Data backup is an essential part of a maintenance plan for any computer system because it is the only remedy against data loss in case of disaster. The fundamental requirement to backup is keeping data integrity. When online data are backed up, both production and backup software access data in parallel and so parts of source data can be modified before they have been backed up. So that backup software should provide coherency of backed up online data in order to supply backup integrity.
There several methods have been invented to provide data coherency in online backup, from which the most advantageous are ones based on the concept of “snapshots”. A snapshot is generally a virtually independent logical copy of data storage (e.g. a volume, a disk, a database or the like) at a particular instant in time. There are file system level and block level implementations of snapshots, yet hereafter only the block-level snapshots are discussed because of their universality and therefore more convenience for general-purpose backup solutions.
Once a snapshot has been created, it can be exposed as a virtual read-only storage whose data can be accessed via standard I/O functions. As soon as a snapshot has been created, production software continues working with production storage while snapshotted data are commonly used for various maintenance tasks such as backup, replication, verification et cetera. There multiple principles of snapshot operation have been contrived. Their common characteristics are (a) use of extra space to preserve snapshotted data and (b) computing overheads imposed by snapshot management means during snapshot creation, operation or deletion, depending on particular technique.
Modern computer systems manipulate storages of very high capacity, and so it is reasonable that those online backup solutions are desired, which provide shorter backup window, produce less impact on performance and require less resources to operate. The present invention proposes an improvement of performance characteristics of a snapshot and so related materials are discussed here.
Principles of snapshot operation are commonly classified into “full copy” and “differential” snapshots. The most known representative of full copy snapshot methods is the “split-mirror” technique. This technique implies continuous mirroring of production storage to a secondary one at normal operation mode. When a snapshot is created, the mirroring is stopped and the second copy of data is split and used as independent data storage. Before a split-mirror snapshot can be re-used again, it needs the mirror re-synchronization has to be made. Software implementations of split-mirror snapshots impose computing overheads for the whole duration of the data mirroring process.
Differential snapshots are based on the idea of holding only the difference between the current data and point-in-time data corresponding to the moment of snapshot creation. The most known representatives of differential snapshot methods are “copy-on-write” (abbreviated to COW) and “redirect-on-write” (abbreviated to ROW) techniques.
The COW technique makes a copy of original data only at the first time data are updated. No data are stored at the moment of snapshot creation yet a snapshot manager starts monitoring I/O writes on production storage. Once controlled data are to be updated the snapshot manager suspends an update operation, stores original data in an auxiliary storage and then resumes data update. If snapshot contents are requested, the snapshot manager takes unchanged pieces of data from a production storage while for changed pieces their original contents are retrieved from an auxiliary storage. At the deletion of a COW snapshot an auxiliary storage is abandoned and nothing is made on production storage. COW snapshots require no recycling period and, multiple COW snapshots can coexist at same time without affecting each other. The U.S. Pat. No. 5,649,152 issued on Jul. 15, 1997 represents an elaborate development of differential snapshot techniques based on COW technique.
The ROW technique exploits the complement idea to COW: updated data are placed to a new location (e.g. are copied to an auxiliary storage) while original data remain intact on a storage. A snapshot manager virtualizes access to production storage in order to accurately redirect I/O requests. At the deletion of a ROW snapshot all accumulated updates should be merged to an original production storage and this procedure may require noticeable time to complete. A complicated management system is required for maintaining multiple ROW snapshots.
Time required for a snapshot re-cycling may play a decisive role in backup solutions that include a stage of data integrity verification. Under these conditions a COW snapshot-based backup solution may appear a preferred choice between other techniques. On the other hand, a ROW snapshot holds a source production storage frozen in a consistent state corresponding to a moment of snapshot creation, during the backup operation. In case a disaster occurred not destroying storage devices before the backup operation has been finished, a source production storage appear in a ready-to-work consistent state as soon as a system is revived and returned into normal operation.
The advantage of full copy snapshots is that they instantly create a really independent copy of original data, which keep snapshotted data integrity regardless of production system function. The disadvantages are very high storage expenses and prolonged recycling period. Until ultimately required by a field of production application a backup solution based on full copy snapshots is too expensive.
Differential snapshots take much less resources and require short or no recycling period in contrast to full copy snapshots. Their disadvantage is that integrity of differential snapshots is dependent on integrity of source data. In case a source storage fails, a differential snapshot of that storage fails as well. Therefore in the view of post-fault data recovery differential snapshots can only be used as auxiliary means for online data backup.
In contrast to full copy snapshots, differential snapshots require an auxiliary storage to keep only updated data, which is usually much less than entire storage capacity. However in worst case a differential snapshot would require an auxiliary storage to have same capacity as the source data storage.
An improvement of characteristics of differential snapshots has been proposed in the U.S. Pat. No. 5,274,807 issued at Dec. 28, 1993 where it is proposed to store only blocks of the source storage containing data while unused blocks should be ignored. In present-day snapshot technologies it is a standard technique applied in volume-based differential snapshots. This technique eliminates unnecessary acts of blocks preservation from snapshot activity and further reduces requirements to auxiliary storage capacity. Now in worst case a differential snapshot would require an auxiliary storage to have capacity be enough to store all used blocks presented on the source storage at the moment of snapshot creation. For nowaday computer systems such estimation of auxiliary storage capacity is very expensive as well.
Another aspect of the problem is that a general-purpose snapshot may include data that actually are not required to be snapshotted from the point of view of an application that has created and uses a snapshot. However prior art principles used in snapshot operation impede exclusion such data from the preservation area of a snapshot. A snapshot creation procedure is commonly works in the following way: when a snapshot is created, new I/O writes are suspended, uncommitted I/O writes are flushed, then a bitmap of used blocks is acquired from a file system service, the COW or ROW technique is started and finally, a normal I/O activity is resumed. A period of time when I/O writes are suspended is commonly a brownout period for a production system, because production software cannot work with its normal performance. In modern systems the snapshot creation period lasts only few seconds or even less because the query of a bitmap is commonly a very fast operation. However, a routine of searching of unnecessary files and determining their location would take a long while, so that it is not used in prior art snapshot technologies. Instead, a snapshot protects unnecessary files together with necessary ones, and so takes more resources and imposes more overheads than is really needed.
Further reduction of requirements to an auxiliary storage capacity is commonly unavailable for general-purpose differential snapshots without a possibly to change snapshot n ability characteristics after it has been created. The further analysis discloses what kind of functionality should be added a snapshot operation.
After a differential snapshot has been created, and backup process has been started, the snapshot imposes extra impact on a host system performance in two cases: first, when a piece of source data is first time updated since the moment of snapshot creation and second, when a piece of data has been read from the snapshot and it turns out to be retrieved from an auxiliary storage of the snapshot. Both described events have probabilistic behavior. Unfortunately their probability cannot be reduced within the frame of prior art snapshot based technologies.
A snapshot is presumed to function for indefinite period of time and not to change its contents within that period of time regardless of current progress of a backup process and more generally, regardless of how snapshotted data are used. In worst case a differential snapshot may require an auxiliary storage to hold a copy of all source data and, when snapshotted data are read from the snapshot, every piece of data will be searched and retrieved from the auxiliary storage.